Apparatus for performing loop-filtering and additional filtering on pixels of an image

ABSTRACT

In one embodiment, loop-filtering is performed by a decoder on four successive pixels if obtained quantization information for a first block is greater than the threshold value and the type of the first block and a second block is intra-coded block type. Two pixels of the four successive pixels are in the first block and two pixels of the four successive pixels are in the second block, which is adjacent the first block. The decoder is also configured to perform additional filtering on two successive pixels on the block boundary based on a comparison result of the quantization information and a second threshold value.

FOREIGN PRIORITY INFORMATION

The present invention claims priority under 35 U.S.C. 119 on KoreanApplication No. 2001-33953 filed Jun. 15, 2001; the contents of whichare hereby incorporated by reference in their entirety.

DOMESTIC PRIORITY INFORMATION

This is a divisional of U.S. application Ser. No. 11/097,394 filed Apr.4, 2005, now U.S. Pat. No. 7,711,054 which is a divisional of U.S.application Ser. No. 10/170,999 filed Jun. 14, 2002, now issued as U.S.Pat. No. 7,272,186; the contents of all of which are hereby incorporatedby reference in their entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a moving picture compression technique,and more particularly, to a loop filtering method in a video coder.

2. Description of the Background Art

Generally, it has been proved that the “H.26L” system developed as anext generation moving picture compression technique is superior to the“H.263” and “MPEG4” systems in performance.

The “H.26L” system as the next generation moving picture compressionsystem differs from the conventional H.263 and MPEG4 in using 4×4 blockbased transformation & encoding, carrying out motion estimation &compensation of a transformation block size, and using a single variablelength coder (VLC).

Superior to the conventional motion picture standards in aspect ofperformance, the H.26L system has excessive calculation of encoder aswell as uses 4×4 block based transformation so that a blocking dominatesthan the conventional standards.

In order to remove the blocking, filters include a loop filtering systeminside the encoder and decoder, a post filter processed in a next stageof the decoder, and a system adding the loop filtering system to thepost filter.

When the blocking is removed by the loop filtering system in general, adifference between an input video and a motion video is affected so thatan encoding rate increases in a specific case. Yet, as the loopfiltering system is installed inside a video coder, complexity of theencoder increases.

As mentioned in the above explanation, the H.26L system according to therelated art requires excessive quantity of calculation despite thesuperiority in performance to the moving picture standards as well ashaving the blocking more dominant than that of the moving picturestandards due to the 4×4 block based transformation system.

Moreover, if the blocking is removed by the loop filtering systemaccording to the related art, the difference between the input video andthe motion video is affected so as to increase the encoding rate in aspecific case. Yet, the loop filtering system built inside the videocoder increases the complexity of the encoder.

SUMMARY OF THE INVENTION

The present invention relates to an apparatus for performingloop-filtering and additional filtering on pixels of an image.

In one embodiment, the apparatus includes a decoder configured to obtainquantization information of a first block including the pixel to befiltered, obtain block types of the first block and a second block, andcompare the quantization information and a first threshold value. Thesecond block is adjacent to the first block. The decoder is configuredto perform loop-filtering on four successive pixels if the quantizationinformation is greater than the threshold value and the type of thefirst and second block is intra-coded block type. Two pixels of the foursuccessive pixels are in the first block and two pixels of the foursuccessive pixels are in the second block. The decoder is configured toperform additional filtering on two successive pixels on the blockboundary based on a comparison result of the quantization informationand a second threshold value.

It is to be understood that both the foregoing general description andthe following detailed description of the present invention areexemplary and explanatory and are intended to provide furtherexplanation of the invention as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this application, illustrate embodiment(s) of the invention andtogether with the description serve to explain the principle of theinvention. In the drawings:

FIG. 1 illustrates a flowchart of a loop filtering process according toan embodiment of the present invention;

FIGS. 2A and 2B illustrate block diagrams indicating block locations inhorizontal and vertical directions according to an embodiment of thepresent invention, respectively;

FIG. 3 illustrates a diagram of pixel locations of adjacent blocksaccording to an embodiment of the present invention; and

FIG. 4 illustrates a table of LQP values for determining DQP(QP) valuesaccording to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE EXAMPLE EMBODIMENTS

Reference will now be made in detail to the example embodiments of thepresent invention, which are illustrated in the accompanying drawings.

Hereinafter, a loop filtering method in a video coder according to anexample embodiment of the present invention carries out a real-timeprocessing of removing a blocking and a ringing phenomenon in accordancewith a pixel location, a coded block pattern, a quantization step size,and a motion vector size in a digital video appliance using the H.26Lmoving picture compression technique, thereby enabling to improve avideo quality of a compression image demanding a low bit rate orhigh-speed processing.

First, explained in detail is a real-time processing technique ofremoving “blocking” and “ringing” by predicting an original video forcorrelation between pixels in accordance with a quantization variableQP, a coded block pattern, and a motion vector size available for adecoder (not shown in the drawing).

When an original video f is transmitted as a compressed form, a videorestructured in a decoder can be expressed by the following Formula 1-1.g=f+n,  [Formula 1-1]where g, f, and n are a compression video rearranged in a scanningorder, an original video, and a column vector of quantization error,respectively.

A relation between respective pixels can be expressed by the followingFormula 1-2 in order to process the Formula 1-1 by pixel unit.g(i,j)=f(i,j)+n(i,j),  [Formula 1-2]where i and j indicate a location of a pixel in vertical and horizontaldirections.

An additional function can be defined as the following Formula 1-3 whenreliability of original pixels and 1-dimensional smoothing function foreach pixel are used together with a regularization. Firstly, ahorizontal direction is considered.M{f(i,j)}=M _(p) {f(i,j)}+M _(N) {f(i,j)},  [Formula 1-3]where M_(p) and M_(N) are functions representing reliability smoothingdegrees of left and right directions for a pixel f(i,j), respectively.

And, such functions are defined as the following Formula 1-4, where theM_(P) and M_(N) are the functions considering f(i,j) & f(i,j−1) andf(i,j) & f(i,j+1), respectively.M _(P) {f(i,j)}=a _(P) {f(i,j)−f(i,j−1)}²+(1−a _(P)){g(i,j)−f(i,j)}²M _(N) {f(i,j)}=a _(N) {f(i,j)−f(i,j−1)}²+(1−a_(N)){g(i,j)−f(i,j)}²  [Formula 1-4]

Each of the first terms of right sides in Formula 1-4 indicatesirregularity with an adjacent pixel, i.e. a difference from an adjacentpixel, for the pixel f(i,j), and each of the second terms indicates areliability for the original pixel f(i,j). Moreover, a_(P) and a_(N)mean regularization parameters representing ratios between irregular andreliability terms defined in the functions, respectively. And, a_(P) anda_(N) are parameters adjusting a relative coherence between the twoterms in Formula 1-4.

With the above manner, the additional function is defined for each pixelof the moving picture, and the function in a vertical direction isdefined by varying the parameter i instead of the parameter j in Formula1-4.

Thereafter, if a differential is applied to the additional functions inFormula 1-4 defined for each pixel of the moving picture in order tofind a restored video from which the blocking and ringing are removed,the following Formulas 1-5 and 1-6 are attained.

$\begin{matrix}{\frac{{\partial M}\left\{ {f\left( {i,j} \right)} \right\}}{\partial{f\left( {i,j} \right)}} = {{\frac{{\partial M_{P}}\left\{ {f\left( {i,j} \right)} \right\}}{\partial{f\left( {i,j} \right)}} + \frac{{\partial M_{N}}\left\{ {f\left( {i,j} \right)} \right\}}{\partial{f\left( {i,j} \right)}}} = 0}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}5} \right\rbrack\end{matrix}$

From Formula 1-5, the following Formula 1-6 is found.

$\begin{matrix}{{\frac{{\partial M_{P}}\left\{ {f\left( {i,j} \right)} \right\}}{\partial{f\left( {i,j} \right)}} = {{2{\partial_{P}\left\{ {{f\left( {i,j} \right)} - {f\left( {i,{j - 1}} \right)}} \right\}}} - {2\left( {1 - \alpha_{P}} \right)\left\{ {{g\left( {i,j} \right)} - {f\left( {i,j} \right)}} \right\}}}}{\frac{{\partial M_{N}}\left\{ {f\left( {i,j} \right)} \right\}}{\partial{f\left( {i,j} \right)}} = {{2{\partial_{N}\left\{ {{f\left( {i,j} \right)} - {f\left( {i,{j - 1}} \right)}} \right\}}} - {2\left( {1 - \alpha_{N}} \right)\left\{ {{g\left( {i,j} \right)} - {f\left( {i,j} \right)}} \right\}}}}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}6} \right\rbrack\end{matrix}$

From Formulas 1-5 and 1-6, the pixel to be restored in a horizontaldirection can be found by the following Formula 1-7.

$\begin{matrix}{{f\left( {i,j} \right)} = \frac{\begin{matrix}{{\left( {2 - \alpha_{P} - \alpha_{N}} \right){g\left( {i,j} \right)}} +} \\{{\alpha_{P}{f\left( {i,{j - 1}} \right)}} + {\alpha_{N}{f\left( {i,{j + 1}} \right)}}}\end{matrix}}{2}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}7} \right\rbrack\end{matrix}$

As the regularization parameters in Formula 1-7 are between “0” and “1”,Formula 1-7 can be defined by the following Formula 1-8.

$\begin{matrix}{{f\left( {i,j} \right)} = \frac{\begin{matrix}{{\left( {2 - \alpha_{P} - \alpha_{N}} \right){g\left( {i,j} \right)}} +} \\{{\alpha_{P}{g\left( {i,{j - 1}} \right)}} + {\alpha_{N}{g\left( {i,{j + 1}} \right)}}}\end{matrix}}{2}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}8} \right\rbrack\end{matrix}$

Looking into Formula 1-8, a blocking-removed video of the compressionvideo is determined by the two, left and right pixel values and theregularization parameters by taking the i_(th) and j_(th) videos asreferences.

As the two, left and right pixel values can be used for encoding anddecoding, two regularization parameter values are set up for attainingthe restored video. For this, “set theoretic” is applied to therestoration by pixel unit so as to represent regularization coefficientsby the following Formula 1-9, where “set theoretic” is a theory forcontrolling the parameters a_(P) and a_(N) in Formula 1-4.

$\begin{matrix}{{\frac{\alpha_{P}}{1 - \alpha_{P}} = {\frac{\left\{ {{f\left( {i,j} \right)} - {g\left( {i,j} \right)}} \right\}^{2}}{\left\{ {{f\left( {i,j} \right)} - {f\left( {i,{j - 1}} \right)}} \right\}^{2}} \approx \frac{K_{P}{QP}^{2}}{\left\{ {{g\left( {i,j} \right)} - {g\left( {i,{j - 1}} \right)}} \right\}^{2}}}},{\frac{\alpha_{N}}{1 - \alpha_{N}} = {\frac{\left\{ {{f\left( {i,j} \right)} - {g\left( {i,j} \right)}} \right\}^{2}}{\left\{ {{f\left( {i,j} \right)} - {f\left( {i,{j + 1}} \right)}} \right\}^{2}} \approx \frac{K_{N}{QP}^{2}}{\left\{ {{g\left( {i,j} \right)} - {g\left( {i,{j + 1}} \right)}} \right\}^{2}}}}} & \left\lbrack {{Formula}\mspace{11mu} 1\text{-}9} \right\rbrack\end{matrix}$

K_(P) and K_(N) in Formula 1-9 are determined statistically as adaptiveparameters depending on a location of a pixel since the irregularitiesbetween pixels at a block boundary and an inner block side are differentfrom each other. And, “QP” is a quantization parameter of a macro blockto which the pixel f(i,j) belongs.

Even though the restored video from which blocking is removed usingFormulas 1-8 and 1-9, the value of the regularization parameter requiresa floating-point operation so as to become a problem in aspect ofcalculation quantity of the filter to remove the “blocking”.

Therefore, Formulas 1-8 and 1-9 are transformed into the followingFormulas 1-10 and 1-11 for integer operation.

$\begin{matrix}{{f\left( {i,j} \right)} = \frac{\begin{matrix}{{\left( {2^{9} - {\alpha_{P} \times 2^{8}} - {\alpha_{N} \times 2^{8}}} \right){g\left( {i,j} \right)}} +} \\{{\alpha_{P}{2^{8} \times {g\left( {i,{j - 1}} \right)}}} + {\alpha_{N} \times 2^{8} \times {g\left( {i,{j + 1}} \right)}}}\end{matrix}}{{2 \times 2^{8}}\;}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}10} \right\rbrack \\{{\beta_{P} = {{\alpha_{P} \times 2^{8}} = \frac{{2^{8} \times K_{P}}{QP}^{2}}{{K_{P}{QP}^{2}} + \left\{ {{g\left( {i,j} \right)} - {g\left( {i,{j - 1}} \right\}}^{2}} \right.}}},{\beta_{N} = {{\alpha_{N} \times 2^{8}} = \frac{{2^{8} \times K_{N}}{QP}^{2}}{{K_{N}{QP}^{2}} + \left\{ {{g\left( {i,j} \right)} - {g\left( {i,{j - 1}} \right\}}^{2}} \right.}}}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}11} \right\rbrack\end{matrix}$

In this case, β_(P) and β_(N), as defined in Formula 1-11, are variablesattained by multiplying a_(P) and a_(N) by 2⁸, respectively. Namely,β_(P) and β_(N) are parameters for making a_(P) and a_(N) into integers.

In an algorithm according to the present invention, the regularizationparameters as integer forms using Formula 1-11 are generated, and thenstored as a look-up table form to be used.

Hence, Formula 1-10 for blocking filtering can be found by knowing thepixel f(i,j) and locations and quantization parameter values of twopixels adjacent to the pixel f(i,j).

Meanwhile, in the loop filtering process of the present invention, avideo of which blocking in a direction of a horizontal axis is removedby loop-filtering a block of a moving picture is attained, and then thevideo is loop-filtered in a direction of a vertical axis so as toprovide the video of which blocking in a direction of a vertical axis isremoved.

FIG. 1 illustrates a flowchart of a loop filtering process according toan embodiment of the present invention.

Referring to FIG. 1, a loop filtering method according to an embodimentof the present invention includes the steps of determining a strength ofa block to be processed in accordance with a coded pattern and aquantization step size of each 4×4 block when a block of a movingpicture is loop-filtered in directions of its horizontal and verticalaxes (S11), carrying out a loop filtering in accordance with thedetermined strength of the corresponding block (S12), and carrying outan additional loop filtering at a boundary region of a macro block sincea relatively big blocking exists at the boundary region of the macroblock (S13). This is explained in detail by referring to FIG. 2A, FIG.2B, and FIG. 3.

FIGS. 2A and 2B illustrate block diagrams indicating block locations inhorizontal and vertical directions according to an embodiment of thepresent invention, respectively, and FIG. 3 illustrates a diagram ofpixel locations of adjacent blocks according to an embodiment of thepresent invention, in which block locations in directions of horizontaland vertical axes are shown.

First of all, the step of determining the block strength St and afiltering state is explained as follows.

The step of determining the block strength St and a filtering state is aprocess for determining the H.26L system as standards, for which TML(test model long-term number 5) is in progress. In this case, the TMLcompresses a video by taking a 4×4 block as a reference.

The TML (test model long-term) moving picture compression system uses atechnique of removing spatial and temporal redundancy information froman encoder so as to transmit spatially & temporally compressedinformation and additional information required for decoding. Of course,in order to remove the redundancy information on a temporal domain, MPEG(moving picture expert group) and H.263 motion compensation systemaccording to the related art are used. The technique of removing theblocking using a basic mechanism of such a TML system is generallytreated by 4×4 block unit.

In this case, H.263 and H.26L are moving picture compression standardsproposed by ITU-T (international telecommunicationsunion—telecommunication).

The system for removing the blocking according to an embodiment of thepresent invention may follow the system of the related art. Yet, thefiltering state is determined by the following pre-treatment steps 1 to3 since the degree of the blocking differs in accordance with a codedblock pattern, a quantization step size, and a motion vector size.

[Step 1]

Strength St of each 4×4 block is initialized by ‘0’(St=0).

[Step 2]

If each of the 4×4 blocks is an intra-coded block or has a non-zerotransform coefficient, St=max(St,2).

[Step 3]

If a motion vector difference between respective motion vectors for thetwo blocks block1 and block2 in FIG. 2( a) and FIG. 2( b) is equal to orgreater than ‘1’, St=Max(St,1).

In this case, max(a,b) means a maximum value of ‘a’ and ‘b’. Namely,max(a,b) is a function for selecting a greater value from ‘a’ and ‘b’.

Moreover, after the strength St of each of the blocks has beendetermined, the loop filtering is carried out in accordance with thequantization size of each of the blocks.

For carrying out the loop filtering, the quantization sizes of the twoblocks block1 and block2 are as follows.

First case is that the quantization value of the block block1 is smallerthan ‘21’ and the strengths of the two blocks block1 and block2 are not‘0’. And, second case is that the quantization value of the block block1is greater than ‘20’ and the strength St of one of the two blocks block1and block2 is not ‘0’.

In this case, the filtering state is determined differently inaccordance with the quantization size because an adaptive treatment isperformed due to the dominating generation of blocking in case of agreat quantization size.

Therefore, when the conditions for the pre-treatment steps 1 to 3 aremet, the loop filtering process is carried out by the following manner.

First, pixel locations of two blocks block1 and block2 are shown in FIG.3. If the above-defined filtering state of each of the blocks issatisfied, values of pixels c, d, e, and f are found by the followingFormula 2-1 (for calculating filter coefficient values to be used inFormula 2-5).r1=reg{St ₁ ,QP,abs(b−c)},r2=reg{St ₁ ,QP,abs(c−d)},r3=reg{max(St ₁+1,St ₂+1),QP,abs(d−e)},r4=reg{St ₂ ,QP,abs(e−f)},r5=reg{St ₂ ,QP,abs(f−g)}  [Formula 2-1]

From Formula 2-1, reg(.) is defined by the following Formula 2-2.

$\begin{matrix}{{{{reg}\left( {{St},{QP},{Diff}} \right)} = \frac{256 \times {M({St})} \times {{DQP}({QP})}}{{{Diff} \times {Diff}} + {{M({St})} \times {{DQP}({QP})}}}},} & \left\lbrack {{Formula}\mspace{14mu} 2\text{-}2} \right\rbrack\end{matrix}$where Diff is a random value and ‘abs’ indicates a function of absolutevalue.

And, M(St) is defined by the following Formula 2-3.M(St=0)=⅙,M(St=1)= 4/16,M(St=2)= 8/16,M(St=3)=1  [Formula 2-3]

And, DQP(QP) is explained by referring to FIG. 4 as follows.

FIG. 4 illustrates a table of LQP values for determining DQP(QP) valuesaccording to an embodiment of the present invention. Namely, DQP(QP) isdefined by the following Formula 2-4, and can be found by substitutingthe LQP values shown in the table of FIG. 4. LQP is a quantization tabledefined in H.26L, and DQP defines a quantization error range inaccordance with the quantization table defined in H.26L as defined byFormula 2-4. Namely, the LQP values are “620”, “553”, “429”, . . . ,“24”, and “22” if the quantization variables QP are “0”, “1”, “2”, . . ., “30”, and “31”, respectively. Namely, as the quantization variable (0,1, 2, . . . , 30, 31) increases sequentially, the LQP value decreaseswith a predetermined value (620, 553, 492, . . . , 24, and 22).

$\begin{matrix}{{{DQP}({QP})} = \frac{676}{{LQP}({QP})}} & \left\lbrack {{Formula}\mspace{14mu} 2\text{-}4} \right\rbrack\end{matrix}$

Formula 1-11 is equivalent to Formula 2-2, and Q^(P2), K_(P)(or K_(N)),and {g(i,j)−g(i,j−1)} in Formula 1-11 are parameters equivalent toDQP(QP), M(St), and Diff in Formula 2-2, respectively.

Hence, the filtered values of the pixels c, d, e, and f are calculatedby the following Formula 2-5 using the defined parameters.c _(mod)={(512−r ₁ −r ₂)×c+r ₁ ×b+r ₂ ×d+256}>>9d _(mod)={(512−r ₂ −r ₃)×d+r ₂ ×c+r ₃ ×e+256}>>9e _(mod)={(512−r ₃ −r ₄)×c+r ₃ ×e+r ₄ ×f+256}>>9f _(mod)={(512−r ₄ −r ₅)×f+r ₄ ×e+r ₅ ×g+256}>>9,  [Formula 2-5]where “>>” means a bit shift, and C_(mod), d_(mod), e_(mod), and f_(mod)mean values of filtering c˜f having the blocking shown in FIG. 3.

Meanwhile, the result from the above-explained processing brings aboutan effect of removing the blocking and ringing with satisfaction. Yet,micro blocking still exists in the boundary region of the macro blockconstituted by 16×16 block unit. In order to settle the micro blocking,an additional filtering, as shown in Formula 2-6, is carried out on thepixel lying at the boundary region of the macro block.d _(mod)=(c _(mod)+13×d _(mod)+2×e _(mod)+8)>>4e _(mod)=(2×d _(mod)+13×e _(mod) +f _(mod)+8)>>4,if QP>16 and d and e are macro block boundary pixels.

Accordingly, the present invention carries out a real-time processing ofremoving a blocking and a ringing phenomenon in accordance with a pixellocation, a coded block pattern, a quantization step size, and a motionvector size in a digital video appliance using the H.26L moving picturecompression technique, thereby enabling to improve a video quality of acompression image demanding a low bit rate or high-speed processing.

The foregoing embodiments are merely exemplary and are not to beconstrued as limiting the present invention. The present teachings canbe readily applied to other types of apparatuses. The description of thepresent invention is intended to be illustrative, and not to limit thescope of the claims. Many alternatives, modifications, and variationswill be apparent to those skilled in the art.

1. An apparatus for removing a blocking artifact using quantizationinformation, comprising: a decoder configured to obtain quantizationinformation related to at least one of a first block and a second block,the second block being adjacent to the first block, the decoder furtherconfigured to compare a value related to the quantization informationwith a constant value, determine whether the first block and the secondblock are intra-coded blocks, perform a filtering process based on thecomparing step if both of the first block and the second block areintra-coded blocks, the filtering process including, calculating valuesof at least two pixels in the first block to obtain filtered valuesbased on pixel values of the at least two pixels in the first block, andwherein the decoder further configured to, filter at least twosuccessive pixels on a block boundary between the first and secondblocks.
 2. The apparatus of claim 1, wherein the filtering processfurther includes: calculating values of at least two pixels in thesecond block to obtain other filtered values based on pixel values ofthe at least two pixels in the second block.
 3. An apparatus forremoving a blocking artifact using quantization information, comprising:a decoder configured to determine whether a first block and a secondblock are intra-coded blocks, the second block being adjacent to thefirst block, the decoder further configured to obtain quantizationinformation related to at least one of the first and second blocks,compare a value related to the quantization information with a constantvalue, perform a filtering process for the first and second blockscorresponding to the intra-coded blocks, based on a result from thecomparing, the filtering process including, calculating values of atleast two pixels in the first block to obtain filtered values based onpixel values of the at least two pixels in the first block, andcalculating values of at least two pixels in the second block to obtainother filtered values based on pixel values of the at least two pixelsin the second block, and the decoder further configured to, filter atleast two successive pixels on a block boundary between the first andsecond blocks.